Search for: All records

Abstract In this study, we used Epistemic Network Analysis (ENA) to represent data generated by Natural Language Processing (NLP) analytics during an activity based on the Knowledge Integration (KI) framework. The activity features a web-based adaptive dialog about energy transfer in photosynthesis and cellular respiration. Students write an initial explanation, respond to two adaptive prompts in the dialog, and write a revised explanation. The NLP models score the KI level of the initial and revised explanations. They also detect the ideas in the explanations and the dialog responses. The dialog uses the detected ideas to prompt students to elaborate and refine their explanations. Participants were 196 8th-grade students at a public school in the Western United States. We used ENA to represent the idea networks at each KI score level for the revised explanations. We also used ENA to analyze the idea trajectories for the initial explanation, the two dialog responses, and the final explanation. Higher KI levels were associated with more links and increased frequency of mechanistic ideas in ENA representations. Representation of the trajectories suggests that the NLP adaptive dialog helped students who started with descriptive and macroscopic ideas to add more microscopic ideas. The dialog also helped students who started with partially linked ideas to keep linking the microscopic ideas to mechanistic ideas. We discuss implications for STEM teachers and researchers who are interested in how students build on their ideas to integrate their ideas. 

Abstract To promote a justice‐oriented approach to science education, we formed a research‐practice partnership between middle school science teachers, their students, curriculum designers, learning scientists, and experts in social justice to co‐design and test an environmental justice unit for middle school instruction. We examine teacher perspectives on the challenges and possibilities of integrating social justice into their standards‐aligned science teaching as they participate in co‐design and teach the unit. The unit supports students to investigate racially disparate rates of asthma in their community by examining pollution maps and historical redlining maps. We analyze interviews and co‐design artifacts from two teachers who participated in the co‐design and taught the unit in their classrooms. Our findings point to the benefits of a shared pedagogical framework and an initial unit featuring local historical content to structure co‐design. Findings also reveal that teachers can share similar goals for empowering students to use science knowledge for civic action while framing the local socio‐political factors contributing to the injustice differently, due in part to different institutional supports and constraints. Student interviews and a pre/postassessment illustrate how the unit facilitated students' progress in connecting socio‐political and science ideas to explain the impacts of particulate matter pollution and who is impacted most. Analyses illuminate how teachers' pedagogical choices may influence whether and how students discuss the impact of systemic racism in their explanations. The findings inform refinement of the unit and suggest supports needed for co‐design partnerships focused on integrating social justice and science. 

We explored how Natural Language Processing (NLP) adaptive dialogs that are designed following Knowledge Integration (KI) pedagogy elicit rich student ideas about thermodynamics and contribute to productive revision. We analyzed how 619 6-8th graders interacted with two rounds of adaptive dialog on an end-of-year inventory. The adaptive dialog significantly improved students’ KI levels. Their revised explanations are more integrated across all grades, genders, and prior thermodynamics experiences. The dialog elicited many additional ideas, including normative ideas and vague reasoning. In the first round, students refined their explanation to focus on their normative ideas. In the second round they began to elaborate their reasoning and add new normative ideas. Students added more mechanistic ideas about conductivity, equilibrium, and the distinction between how an object feels and its temperature after the dialog. Thus, adaptive dialogs are a promising tool for scaffolding science sense-making. 

Abstract Guiding teachers to customize curriculum has shown to improve science instruction when guided effectively. We explore how teachers use student data to customize a web-based science unit on plate tectonics. We study the implications for teacher learning along with the impact on student self-directed learning. During a professional development workshop, four 7th grade teachers reviewed logs of their students’ explanations and revisions. They used a curriculum visualization tool that revealed the pedagogy behind the unit to plan their customizations. To promote self-directed learning, the teachers decided to customize the guidance for explanation revision by giving students a choice among guidance options. They took advantage of the web-based unit to randomly assign students (N = 479) to either a guidance Choice or a no-choice condition. We analyzed logged student explanation revisions on embedded and pre-test/post-test assessments and teacher and student written reflections and interviews. Students in the guidance Choice condition reported that the guidance was more useful than those in the no-choice condition and made more progress on their revisions. Teachers valued the opportunity to review student work, use the visualization tool to align their customization with the knowledge integration pedagogy, and investigate the choice option empirically. These findings suggest that the teachers’ decision to offer choice among guidance options promoted aspects of self-directed learning. 

We report on design-based research to refine a professional development workshop that supports teachers to customize online curricula. We iteratively design representations to make the knowledge integration pedagogy of the curricula visible. We study ways to make the work of students using the curricula actionable for participating teachers. We analyze participants’ trajectories across the three iterations of the workshop. Initially, when participants realized they could customize the online curriculum, they developed feelings of ownership. Then, as participants deepened their understanding of the pedagogy, they began to use it to evaluate their own instruction. The trajectory culminated in participants connecting the pedagogy to student work from their own classroom. This led to a shift from focusing on remedies for misconceptions to seeking opportunities for building on students’ nascent ideas when customizing. The workshop refinements empowered teachers to mobilize the pedagogy to interpret their students' work to inform their customization decisions. 

Recent work on automated scoring of student responses in educational applications has shown gains in human-machine agreement from neural models, particularly recurrent neural networks (RNNs) and pre-trained transformer (PT) models. However, prior research has neglected investigating the reasons for improvement – in particular, whether models achieve gains for the “right” reasons. Through expert analysis of saliency maps, we analyze the extent to which models attribute importance to words and phrases in student responses that align with question rubrics. We focus on responses to questions that are embedded in science units for middle school students accessed via an online classroom system. RNN and PT models were trained to predict an ordinal score from each response’s text, and experts analyzed generated saliency maps for each response. Our analysis shows that RNN and PT-based models can produce substantially different saliency profiles while often predicting the same scores for the same student responses. While there is some indication that PT models are better able to avoid spurious correlations of high frequency words with scores, results indicate that both models focus on learning statistical correlations between scores and words and do not demonstrate an ability to learn key phrases or longer linguistic units corresponding to ideas, which are targeted by question rubrics. These results point to a need for models to better capture student ideas in educational applications. 

The effectiveness of using learning analytics for learning design primarily depends upon two concepts: grounding and alignment. This is the primary conjecture for the study described in this paper. In our design-based research study, we design, test, and evaluate teacher-facing learning analytics for an online inquiry science unit on global climate change. We design our learning analytics in accordance with a socioconstructivism-based pedagogical framework,called Knowledge Integration, and the principles of learning analytics Implementation Design. Our methodology for the design process draws upon the principle of the Orchestrating for Learning Analytics framework to engage stakeholders (i.e. teachers, researchers, and developers). The resulting learning analytics were aligned to unit activities that engaged students in key aspects of the knowledge integration process. They provided teachers with actionable insight into their students’ understanding at critical junctures in the learning process. We demonstrate the efficacy of the learning analytics in supporting the optimization of the unit’s learning design. We conclude by synthesizing the principles that guided our design process into a framework for developing and evaluating learning analytics for learning design.